LARGE DEVIATIONS ASYMPTOTICS FOR LARGE WAITING 

TIMES 
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Abstract. In this paper we investigate the statistics of large waiting times 
(with respect to the total waiting time) for Bernoulli processes. We deter- 
mine the corresponding rate functions explicitly and prove a large deviations 
asymptotic. By this we have estabished a large deviation principle for which 
the rate function is not the Legendre transform of some free energy function. 



1. Introduction and statement of Main results 

We will investigate a large deviation principle for return time processes defined over 
Bernoulli processes. Let (a>j) be a Bernoulli process over the symbol space {1, . . . , r} 
with probability vector (pi,P2, ■ ■ ■ ,Pr), that is, for all (fei, . . . , k n ) € {1, . . . , r} n and 
n g N, we have P (w^ = h,.. .,u in = k n ) = n"=iPfc«- 

Now, let us fix k g {1, . . . , r} with p := p^. € (0, 1) and set the first hitting time 
Z\ (u>) to k to be the first occurrence of k in the sequence u> — (u>i), i.e. 

Zi(uj) := inf{n g N : n > 0, uj n = k}. 

Recursively, we define the n-th total waiting time Z n (u>) to be the n-th occurrence 
of the symbol k in the sequence (u>j), i.e. 

Z n (uj) := inf{£ g N : t > Z n _i (w) ,w e = k},n> 2. 

Let us set Zq = 0. Then the n-th waiting time z n (w) between two successive 
occurrences of the symbol k after the n-th occurrence of k is given, for n £ N, by 

z n (u) := Z n (uS) - Z„_i(w). 

It is well known that in this situation the (z n ) are i.i.d. random variables and that 

E(zi) =p~\ 

In this paper we are going to study the process (z n +i/Z n ). Clearly, this process 
converges to in probability (even almost surely). Our aim is to prove a large 
deviation principle for this process allowing us to estimate the probability of large 
waiting times with respect to the total waiting time. More precisely, for q > we 
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will show that the following limit 

limilogpf^tl >q \ = _ J(g) 
n n \ Z n J 

exists and that the rate function / can be determined explicitly. Interestingly, it 
will turn out that the rate function / is not convex and hence / is not the Legendre 
transform of a free energy function (see Remark [5j. 

To state our main theorem we need to fix some notation. We say that the two pos- 
itive real sequences (ct„) ngN , (fo n ) ngN are comparable, a n x b n , with comparability 
constants c\ > and ci > if for all n € N we have c\a n < b n < C2a n . 

Theorem 1 (Large Deviations Asymptotic). With the notation from above we have 

<" : w-)K E ^ U!,(i - p)C(it, i ■ 

where C := C p (q) := ^(1 — p) q 1 — 1^ and the comparability constants may be 
chosen to be 1 — p and 1. 

Remark 2. We will see in the proof of Theorem [l] that for q g N a stronger asymp- 
totic holds. Namely, 

'(-^>«)~(^< 1 -^)"- 

where a n ~ b n means that limj^oo a n /b n — 1 for some positive sequences (a„) and 
(b n ). 

As an immediate consequence we obtain a large deviation deviation principle. For 
this let us recall the Large Deviation Principle (see e.g. |E1185l IDZ98J). 

Definition 3. Let H be a complete separable metric space, B (TL) the Borel a- 
algebra of 7i, and (Q n ) n( z^ a sequence of Borel probability measures on B(TL). We 
say (Q n ) fulfills a Large Deviation Principle with normalising sequence (o n ) JlgN € 
M5 ( lim a n —> oo) and rate function I : TL — > [0, oo] if the following properties are 
fulfilled. 

(1) / is lower semi-continuous on 7i, 

(2) / has compact level sets, 

(3) limsup a" 1 In Q n {K} < — mf X £K I(x) for all closed sets K C TL, 

n — >oo 

(4) liminf a" 1 lnQ„{G} > — inf xS G I(x) for all open sots G C TL. 

n — >oo 

Corollary 4. Let Q„ be the distribution of z n+ \/ Z n with respect to the Bernoulli 
measure P, i.e. Q n (A) — P (w : z n +\(u)) / Z n (uS) € A). Then {Q n ) n c® fulfills the 
Large Deviation Principle with normalising sequence N and rate function L p given 
for q > by 

I p (q) :=-(C+ 1) log (C + 1) + Clog C - \ogp - ((1 + q)C + q) log(l - p) 
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and, for q < 0, by I p (q) = +00. The rate function I p is strictly increasing and 
concave on (0,oo) (see Fig. 1 1. l\ for a parametric plot of I p for different values of 
the parameters p G (0, 1)). 




Figure 1.1: A parametrical plot of the rate function I p for different parameters 

pe(0,l). 



Remark 5. At this point we would like to recall a classical large deviation law for 
the empirical means of i.i.d. random variables. Let (X n ) be a sequence of i.i.d. 
random variables with finite moments and S n := Y^k=x ^-k- Then by the weak law 
of large numbers we have that the empirical mean S n /n converges in probability 
to E(Ai) =: jU as n tends to infinity. In this situation a large deviation principle 
holds with convex rate function /, that is 

limsup-logP f — > t ) = -I(t), t>fi. 
n n \ n J 

In fact, / is given by the Legendre transform of the Helmholtz free energy function 

H x ■ t h-> E (exp (t ■ Xi)) , 

which is necessary convex. Since in our situation the rate function is not convex it 
cannot be the Legendre transform of a free energy function. More precisely, if we 
set w n := nz n+ i/Z n then a possible candidate for the free Helmholtz energy would 
be 

H w : 1 1— > lim — log E (cxp (t ■ w n )) . 
n— >oo n 

Since I p is not the Legendre transform of H w we conclude by a general Large 
Deviation Theorem of Ellis [E1185 that H does not define a finite and different iable 
function (for a generalisation see also jKesOlj 'l. 
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Remark 6. Furthermore, Fig. 1.2 might suggest that the rate function I p approaches 
an asymptotic line for q tending to infinity. Indeed, since the parameter C p (q) — 
((1 — p)~( 1+ i) — l) 1 vanishes for q — > oo, a straight forward calculation gives 

i p (q) ~ -i ■ M 1 - p) - 

for q tending to infinity. That is, the asymptotic line has slope — log (1 — p) and 



intersects the 7 p -axis in — log (p) (see Fig. 1.5 ^b) 



Remark 7. We would also like to remark that the sets {z n+ i/Z n > q} considered in 
this paper play a crucial role in the fractal analysis of the set of non-differentiability 
of certain singular distribution functions (Devil's staircases) (cf. [Li07| [LXD02] : for 
another account on the connection between fractal geometry and large deviations 
we refer to jDKOlj ). 



The proof of Theorem [T] heavily depends on a generalised version of Laplace's 
Method. Since this statement might be of some interest for itself we will give the 
precise statement at this point. 

Proposition 8 (Generalised Laplace's Methode). For n £ N, let F n <E C 3 (R) 
be a concave function having a unique absolute maximum in x n G K such that 
F"(x n ) < 0. Also suppose that there exists a sequence (r„) ngN £ such that for 

o n := \\ _t?1( — \ and Vn ■= sup{|F"'(£)| : £ £ (x n - r n ,x n + r n )} 
y f n (x n ) 

we have, as n tends to infinity, 

— — * oo and S n := a 2 n r n r] n — » 0. 

0~n 
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Let I C K be an interval such that for all n € N large enough we have (x n — r„, + 
r n ) C /. TTierc 



See Remark 13 for corresponding one-sided versions of this theorem. 



The paper is organised as follows. In Section [2] we give a proof of Corollary [4] and 
Theorem [l] and in Section [3] we will prove the generalised Laplace's Method. The 
Appendix is devoted to some special functions and their basic properties which are 
crucial in the proof of Theorem [I] 



2. Proof of the large deviation laws 



2.1. Proof of Corollary g} 



Proof of Corollary^ The fact that I p is the rate function as claimed in the corol- 
lary follows immediately form Theorem [T] The first derivative of I p is given by 

I'M - - ( (1 „ pr U)^ + l ) ln C 1 - p) > ^ 

which is strictly positive on (0, oo). Therefore, the rate function is strictly increasing 
on (0,oo). For the second derivative we have 

1 / ln(l-p) ^ 1 



which is strictly negative on (0, oo) showing the concavity of I p . □ 



2.2. Proof of Theorem [T} Theorem [T] is an immediate consequence of the fol- 
lowing lemmas and Theorem [8] For the definitions of the Beta Function B and 
the Polygamma Function ipk, k € No, we refer the reader to the Appendix of this 
paper. 

Lemma 9. For the large deviation probability we have 

)oc 
x]T.9(n)e^\ 

where f n : x i— > — ln B{x + 1, n — 1) + x(l + (?) ln(l — p) and g„ := p — , n G N 
and the comparability constants can be chosen to be 1 — p and 1. 
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Proof. For x S R let |_a;J denote the largest integer not exceeding x. Using the 
definition (4.2 I of the Beta Function as introduced in the Appendix we have 



£ ( „-W-„/ ' + " (1 -'" t " , ( £ 

V (fe- 1 )' p f^-lVl _ p) *-n (l-P) LfcgJ 

4- (n-l)!(fc-n)! p P 

V ( fc + n ~ 1 ) ! n/, _ „^+L(fc+n)?J 
A, (rz-l)!(fc)! P 1 W 

E (fe + n - 1)! , _ sfc+(fe +n ) g 
;=o (rz-l)!(fc)! P 1 W 

£j(„-l)r(n-l)r(A + l) P 1 p) 



oo 1 
V p n (l - p) fe +9( fe +") 



k=0 

= y^ P ( l ~ V) c -lnB(fc+l,n-l)+fc(l+ g )ln(l-p) /„(k)_ 

In here, the comparability constants can be chosen to be 1 — p and 1. To verify 
Remark [2] we notice that for q G N the comparability may in fact be replaced by 
equality. This proves the Lemma. □ 

Lemma 10. For n > 2 we have that for the maximum x n of 

f n : R+ -> R; x h-> -lnB(.T + l,n- 1) + x(l + g)ln(l -p) 



we /lave x n ~ Cn C = C p (5) = ( (1 — p) q 1 — 1 j as defined in the intro- 
duction. 



-1 



Proof. Using the relations (4.2 I, (4.4 1 and (4.5 I from the Appendix, we have for 
x > 

f n (x) = ((-lnB(x+l,n-l)+x(l + q)]n{l-p))' 
(dB/dx) (x + l,n-l) 1 
= B(x + l,n-l) -d + ^Mj^) 



r'(ar+l) r'(x + n) 

r(x + 1) ~ r(x + n) 



Co 

n-1 



i/>o0 + n) - ^ (a; + 1) - C = V — ■ Co, 
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where Co = — (1 + q) ln(l — p) > 0. Now uniformly in x > we have 

n—l -, r n-l 



E — 

* — ' X + II 

y=l ^ y 



1 

x + y 



dy = In 1 



n- 2 
x- 1 



Hence, the root y„ := C (n — 2) — 1 of a; ^ log (1 + (n — 2) / (x — 1)) — Co is 
asymptotically equal to the root of f' n . 



Using (4.4 1 and (4.5 I we get for the second derivative of /„ in x > 

n— 1 

fix) = ^(x+»)-v»i(x+i) = y;-- — -2<o. 

fc =i ( x + k ) 

This proves the lemma. 

Lemma 11. Let r n := a;^ w«i/i /3 € (1/2, 1), cr n := 1 / \/—.fn(x n ), and x n ~ Cn, 
bGN. Then, for n tending to infinity, we have 

&n ~ • \/C 2 + C and — — * oo. 



□ 



Proof. Using the explicit formula for the second derivative of f n and the fact that 
x n ~ Cn we get 

2 — n • 
(x n +n — l)(x„ + 1), 



1 



1 



V-fnM \Jl (Xn + V) 

~ v 7 ^ • Vc 2 + c. 

In the same way we get for n tending to infinity 

y x n yx n 



On 



+ n) ^JC 2 + C 



□ 



Lemma 12. Let r n := wii/i /3 S (1/2, 1), n G N, and lA n := (x„ — r n ,x n + r„ 
i„ ~ Cn. Then, for n tending to infinity, we have 

r?„:=sup{|/;"(0|:eeW„}xi 



and 5„ := a n r n r] n — > 0. 



Proof. For the third derivative of /„ we have by (4.4 1 and (4.5 I of the Appendix 
uniformly in hi n 



x+n—l _ . 



(a: + y) ; 



1 



1 



(x + l) 2 (x + n-l) 2 ' 

The maximum of \f„\ on the interval U n is obtained in the left boundary point of 
this interval and since j3 < 1 we obtain 

Vn ■= SUp - 7 , n2 - 7 3 - -o- 

£ew„ (»n - r n + 1) {x n - r n + n - iy n z 
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Hence, for n — * oo, we have 



□ 



Now we are in the position to give a proof of our main theorem. 



Proof of Theorem^ Combining the above lemmas for x n the unique maximum of 
fn, r n '■= %n, i£N, for some j3 € (1/2, 1), we deduce from Theorem [8]and Stirling's 
Formula (n! ~ \f2im (n/e) n ) that 



2tt 



^/2irCn(C + 1) + " . ^^ "(l - p )-.+((*.+")«) 
(n-l)!(.T„j! 

f Cn + n- i y c+1 )" /n--l\ Cn ^ n Cn(1+g)+ng 
\ n — 1 / \ Cn J 



Since, 

\ n-l 
we get 



e C (C+1) (C+l)n 



n - 1 
Cn 



C 



Cn 



/>oo 

/ g n e^dx 
Jo 



((C + l) 



c+i 



Due to Lemma we are now left to show that 



p{\ - p )C(i+g)+ 9 j 



./o 



fe=0 



(2.3) 



Since /„ is concave we have that /„ is increasing on (0, x n ) and decreasing on 
(x n ,oo). Consequently, 



L^J-i 



fe=0 



L^nJ+l 



and 



This gives 



OO -oo oo 

£ e^ fc > < / dx < 

II,-. J X„ 71 



fc=|_:r„J+l 



OO /.oo 

fe=0 1/0 



< e /»(0) +e /«(bnj) < 2e /nC»n). 



Since by Theorem [8] and Lemma 11 



oo 



2tt 
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3. Proof of the generalised Laplace's Method 



Proof of Theorem^ Using Taylor's formula we have for every x € (x n — r n ,x n + 
r n ) =: U n that there exists g x E U n such that 



F n (x) = F n (x n ) + 



F n (x n ) 
2! 



(3.1) 



--:G n (x) 



The second order Taylor approximation is asymptotically exact on U n in the sense 
that, for n — > oo, 



sup 



F n (0 (a; - x n f /3! 



1 2 

- sup r n a„ 

6 i&A n 



F^(x n ) (x - x n y /2\ 
This implies that for the integration restricted to U n we have 



1 



0. 



exp I G n (x) + ^^(x - x n )A dx 



>F n (x n ) 



exp 



r n /cr n 



(x - x n y 

2a? 



exp 



t 2 



F n (X - X n ) 

3-F" n (^n) ^n) 



— r n / (T n 

Hence, we are left to show that, for n — > oo, we have 

' f 3 F ^dx^0. 



,F n (x n ) 



dx 



The error term in the second order Taylor approximation on U n is at most 



d n :=-fBup{|JC(0|:£€Za = 



5 r 2 



To find an upper bound for the integrand on the complement of U n we will con- 
sider two straight lines connecting the point (x n ,F n (x n )) with the two points 
(x n ± r„, G n (x n i r n ) + d n ). The absolut value of the slopes of the two lines is 
then given by r n /a n (1/2 + <5„/3) and, since F n is concave, on M\W„ these lines lie 
necessarily above the graph of F n . Hence, for n tending to infinity, we obtain the 
following upper bound for the integral. 



CT„e 



1 

F n (x n ) 



F »<*> dx < 2a- 1 / e -<r n /«l(l/2+S n /3)) dt 
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Combining all of the above, we therefore obtain 
/ e F ^dx = [e F ^ x Ux + [ e F ^ x Ux 

JR J J 



x n -\-r n 



3 F "^ dx 



J e F ^ x Ux\ 



J e F ^)dx 



V 



□ 



Remark 13. If the condition U n C / is not fulfilled for all sufficient large n € N, 
then a one-sided version of Theorem [8] still may hold. 



(1) If I — [a, oo) and 



0, then we have 



e F ^dx~e F ^ x ") / - 



(2) If I = (oo, b] and ^± n ^°° 0, then we ha 



// V -2i^'(^) 

For the proof of these claims one has to consider the one-sided (say right-sided 
for case (1)) Taylor expansion of F n and mainly use the elementary fact that with 

hn x i — ^ - exp ^— | ^ a ~ z " ^ J we have under the assumptions in (1) 

a+r n ^ 

h n (x) dx ~ -. 



4. Appendix 



For a good account on the following facts we refer the reader to |AS64j . 
For positive i£l the Gamma Function is given by 

/>oo 

T(x) := / f-V**, 
Jo 

and for positive a; and ?/ the Beta Function is given by 

T(a; + y) 

For the partial derivative of the Beta Function we have 



(4.1) 



(4.2) 
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where ip n denotes the Polygamma Function defined, for n € N, by 

gn+i 




(4.4) 




(4.5) 
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